The early empiricists na�ly assumed perception to be a fairly simple task of extracting the components of an image and forming them into objects we recognise. Such a view was banished by the rigours of attempting to model such a process computationally � the problem is very much harder. Even from the point that the photoreceptors in the retina are first stimulated, the visual system is extracting and processing the information � a passive image, no matter how rich and detailed, is of little use to us. In order to transform these �tiny, distorted, upside-down images� (Gregory 1966) into the three-dimensional mental constructs we see, we have to construct this visual representation from �unconscious inferences� (Rock, 1984) and ambiguous data.
Zeki first showed that the visual system seems to operate using perhaps three parallel pathways, which can be roughly characterised as being for analysing:
I will attempt to outline the anatomical, psychological and clinical evidence for this claim, as well as the evidence against it, before discussing why the brain might have evolved to segregate the functions in this way.
The anatomical evidence for segregation of function, can be traced to where the three pathways originate, in the division of retinal ganglion cells into the larger M-type cells and the smaller P-type cells, which project to the magno-cellular and parvo-cellular layers of the LGN respectively.
The P-type cells lead to the �what� system for perceiving shape. They have slower, sustained and linear responses (linearity is when the centre and surround exactly cancel each other out, giving no response under uniform illumination). Their slow adaptation makes them capable of high resolution, which is necessary for seeing stationary objects in detail. Moreover, P-cells are colour opponent, i.e. they distinguish between inputs from the three types of cones. This, and their sensitivity to edges and their orientation makes them better for extracting information about borders.
The parvocellular interblob pathway conveys mostly information about form and colour. It arises from the parvo-cellular sub-division of the LGN, projects to layer 4Cb in V1, then to the interblobs of layers 2 and 3, then to the pale stripes of V2, then V4 and finally terminates in the infero-temporal lobe.
The parvocellular blob pathway is specialised for colour, as part of the �what� system. Like the interblob pathway it arises in the LGN, where it projects to the blobs of layers 2+3 in V1, then to the V2 thin stripes, then V4 and terminates in the infero-temporal lobe.
The M-cells will eventually lead to the �where� system. Consequently, they tend to have fast, transient and non-linear responses. Areas of the magno-cellular pathway have large receptive fields to enable them to make inferences (especially about motion) on a larger scale in the visual field.
M-cells are not specifically sensitive to motion themselves. However, their fast, transient responses, though they lower the resolution, are important for signalling temporal variation in contrast, which is eventually transformed in V1 and V5 by neurons that respond to particular directions of motion.
Unlike P-cells, M-cells simply add up the responses of all three cones together, losing the colour information, which is why experiments (see below) often use equiluminance-insensitivity to isolate the magno-cellular pathway (equiluminant stimuli vary in colour, but not in luminance, so if an image was converted to black and white, its equiluminant colours would be indistinguishable greys). This makes them comparatively poor at detecting contours or borders of colour contrast, and analysing stationary objects.
The magnocellular pathway conveys information about motion and three-dimensional spatial relationships. Lesions in the magnocellular system give rise to selective deficits in motion perceptiona and in eye movements directed towards targets (see below). From the M-cells� projection to the magnocellular layer of the LGN, it projects to 4Ca in V1, then to layers 4B and 6, then to the thick stripes in V2, then V3, and finally to MT (a.k.a. V5), MST and other areas in the parietal cortex.
The visual system needs to be able to deal with and distinguish motion of the visual field (i.e. motion of objects in the environment) and motion due to the movement of the head and eyes. I will briefly talk about the neural system for motion, to give an example of how one function (such as motion) can be segregated from the visual input, and how it can be a very effective means of encoding the information (evidenced in its early stages by the twenty or so partial and complete retinotopic maps in the striate and extriate cortices).
Motion in the visual field is analysed by a special neural system. In fact only the evolved primates can respond to objects that do not move. Frogs, for example, cannot even see objects unless they are moving. They do not �see� in the sense that we do � when their tongues flick out to catch flies, it is more like a reflex reaction to small, dark, fast-moving objects. Motion in the visual field is detected by comparing the position of images at different times � the visual system should be able to compare the previous localtion of an object with its current location by extracting the necessary information from the retina. This is complicated by the fact that information about the direction of motion from a small receptive field can be ambiguous. For example, the aperture problem (Movshon, 1990) demonstrates that if a grating of diagonal lines is moved either downwards, sideways or perpendicular to the gratings, then it will always appear to move in the same right-downwards direction � in order to be sure, information from two separate local areas needs to be taken. The difficulties involved increase with more complex objects and surfaces moving in three dimensions. Problems like the aperture problem highlight the need for a more complex solution, prompting researchers like Marr and Movshon to propose that information about motion in the visual field is extracted in two stages. The first stage is concerned with one-dimensional moving objects and measuring the motion of the components of complex objects. The second stage involves higher-order neurons combining and integrating the components of motion analysed by several of the initial stage neurons.
Motion of the head and eyes involves co-ordinating the vestibulo-ocular reflex, the continual micro-saccades that we don�t even notice, and larger voluntary movements of the head so that we don�t misinterpret these as huge movements in the visual world around us.
Ramachandran and Gregory first tried to demonstrate that motion is processed separately from colour by experimenting on our ability to detect motion under equiluminant conditions. They showed that the magnocellular pathway, originating from colour-insensitive M-type retinal ganglion cells, is wholly segregated from the colour-carrying parvo-cellular pathway, because perception of motion disappears at equiluminance. Livingstone and Hubel�s later experiments drew similar conclusions for perspective, relative size of objects, depth, figure-ground relations and visual illusions � all of these disappear at equiluminance, and so also must be mediated by the magnocellular system.
This raises the question of why these relationships should be mediated by the same system alone, since it would not have these problems. I will discuss this below.
Convincing evidence for the segregation of function comes from the effects of damage to local areas of the visual pathways, resulting in a selective loss of function, known as an agnosia. There are agnosias in the recognition and usage of real objects, the recognition of drawn objects, the recognition of faces, naming and distinguishing colours (damage to V4 can result in achromatopsia, the loss of colour vision and imagination) and the objects they are associated with, stereoscopic vision and the discernment of movement (movement agnosia occurs after bilateral damage in the cortex of MT or MST, and results in a selective loss of movement perception without loss of any other perceptual capabilities).
The specificity of these agnosias, whether for form and objects, colour, motion or stereopsis supports the anatomical evidence that the pathways are parallel, and functionally specific. A lesion in one may not affect the other pathways to the same extent, causing these selective symptoms. If the visual system was serial or did not segregate function in this way, we would expect lesions to produce a multiplicity of difficulties, or perhaps, through re-acquisition of function, an overall weakening of the visual system without specific loss of ability.
In fact, Freud first posited that cortical defects might be responsible for the specific inabilities of certain patients to recognise visual objects. This was prompted by the case of one well-studied patient with intact visual fields, who lost all perception of motion and could not distinguish between stationary and moving objects. It is hard to see how this could happen without some degree of localised segregation of function.
There is undoubtedly a great deal of inter-connectedness between the three pathways at every level (Van Essen), and it may be that the segregation is not as clearly delineated as has been suggested. Each visual clue appears to be handled by at least one pathway, to a greater or lesser extent. Binding the three pathways interactively (as opposed to the progressive transformation of information in a serial system) may involve inputs from brain centres known to affect attention, e.g. the pre-frontal cortex, claustrum or the pulvinar.
There is one other point to bear in mind. We know about the individual magno and parvo cells� physiology, and so we might expect that the magnocellular system contains a stream of black and white high contrast information. In fact, the sheer numbers of P-cells increase the role they play in our contrast sensitivity. This is just one example of the dangers of over-simplifying the segregation.
The question behind this entire discussion is �why is function segregated in the visual system�. What benefit is there to having two separate visual systems, which only give rise to our internal visual representation when they come together? There could be two answers, and both of them are based on the fact that we do not evolve to a long-term plan. If the intermediate step does not give us an evolutionary advantage, then it is almost impossible that a mutation large enough will give rise to the jump.
It seems certain that the �where� system is of greater immediate value to us. The frog does not need to be able to recognise the fly, classify it in Latin, and look deeply into its eyes before gulping it down. All it needs to know is that there is a little black dot buzzing in front of it, and exactly where that dot is. The �where� information is more urgent. Knowing what the object is, but having no idea where it is until later, would be of little value to us. Having a parallel magnocellular �where� system may allow us to glean the information with most survival value quickest.
However, the evolution of a �what� system may also have been held back until three preliminary developments made it possible and valuable to the animal: colour vision, greater social complexity and tool-use.
Colour vision helps mainly in identifying non-poisonous food-stuffs and in seeing through camouflage that would be effective in black and white. Furthermore, the fact that it is only in the parvocellular system that colour is present implies that it is less important in determining where something is, but extremely valuable in identifying borders and surfaces for outlining the forms of objects.
The human peculiarity of intellect can not be explained purely in terms of the advantages it gives us in intelligently adapting to our material environment, since the problems of material survival aren�t difficult enough to dictate the need for it. Indeed, our current situation of dominance over every other species on the planet is as much a result of our ability to communicate and store knowledge collectively. Similarly, tool use is one of our hallmarks, but the great pre-historic developments in tools do not correlate with similar great evolutionary developments in our brains. In fact, it seems that it is other people who provide the headache� Our big brains are probably necessary in order to keep a complex human society together. Different species have different neo-cortex ratios, and these relate almost linearly to the size of the social groups that the species form. On this basis, humans have an intelligence which correlates with social groups of about 150, the size of the villages prehistoric African man lived in. In parallel with this, our visual system has had to keep up � knowing who someone is and what their facial expression can tell us, as well as where they are, is vital in a society of any size and complexity. Higher mammals, especially primates, live in the most complex complex social hierarchies and require �what� visual systems to match.
In the same way, our beautiful opposable hands, delicate fingers and wealth of somatosensory information benefits from having a distal sense which can match it. Vision tells us where something is relative to us, and what it is. This is enormously valuable to know, for both animate and inanimate objects.
Marr concluded that vision is the process of discovering from images what is present and where, in the visual world. However, it was not initially apparent that these two functions are segregated, or indeed why this should be so. I have tried to show that there is evidence and good reason to suggest the answers.